Repository navigation
SK-3075: add Detect V2 to flowvault and skyvault SDKs - #286
Open
saileshwar-skyflow wants to merge 12 commits into
Open
saileshwar-skyflow wants to merge 12 commits into
saileshwar-skyflow wants to merge 12 commits into
Conversation
Implement Detect V2 (deidentify_string, deidentify_file, get_run, reidentify_string, reidentify_file, check_guardrails) in the shared `common` package and re-export it from both flowvault and skyvault. common: - detect models, controller, request/response mappers, run polling, file-output writer, and validations under common/vault/detect - V2 detect enums under common/utils/enums/detect.py; V2 detect messages in common SkyflowMessages - behaviour ported from the Java flowvault SDK @ flowvault-release/26.10.1 (exponential poll backoff 1..32, submit-only status None, minimal client-side validation, auto-write of processed files) flowvault: - wire DetectController via make_skyflow_class(detect_cls=...) - build the common detect REST client on VaultClient and expose get_strings_v2_api / get_files_v2_api / get_guardrails_v2_api - re-export skyflow.vault.detect and the V2 enums - detect_api samples, README section, and unit tests skyvault: - replace the removed V1 detect API with V2 (same method/class names) - migrate the hand-written layer to the regenerated client (drop V1 prefix, Skyflow -> SkyflowAuth) and update tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lines - common/skyvault generated rest version.py: guard metadata.version() with try/except PackageNotFoundError (matches flowvault), so importing the package never fails when the distribution metadata is absent. - Refresh flowvault/skyvault public-API contract baselines for the Detect V2 surface (flowvault adds the V2 enums; skyvault replaces V1 detect with V2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Public API contract change (
|
Public API contract change (
|
The common package CI runs tests with `python -m unittest` (no pytest installed), so rewrite common/tests/vault/detect/test_detect.py as a unittest.TestCase (self.assertRaises, tempfile instead of the pytest tmp_path fixture). Runs green under both unittest and pytest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promptless documentation updates
|
skyflow-bharti
approved these changes
Oct 29, 2026
…e and ReidentifySourceType
Mirror the Java Detect V2 contract change: the single DataSourceType enum is split
into DeidentifySourceType (deidentify_file data_source and the output_type of
DeidentifyFileResponse / GetRunResponse) and ReidentifySourceType (reidentify_file
data_source and ReidentifyFileResponse output_type).
Like Java, the two source enums are defined per SDK rather than in common:
- skyvault DeidentifySourceType: BASE64, SKYFLOW_ID, PRESIGNED_URL
- flowvault DeidentifySourceType: BASE64, PRESIGNED_URL (no skyflow id)
- ReidentifySourceType: BASE64, PRESIGNED_URL in both (re-identification never
accepts a skyflow id)
common stays SDK-agnostic: the controller/mappers resolve the response output_type
through the per-SDK VaultClient (get_deidentify_source_type / get_reidentify_source_type),
and the file-output writer compares the output type by value. Updates both SDK enum
re-exports, the reidentify_file sample, and the unit/re-export tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"# DetectController — Detect V2" produced an awkward anchor (#detectcontroller--detect-v2) and didn't match the cross-SDK convention; the Go flowvault README uses "Detect". Renamed to "# Detect" (anchor #detect); the section intro already states it is Detect V2 and how to get the controller. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3076 add v1-deprecation warning * SK-3076 fix ci-check
The 865a4f5 regeneration renamed generated REST client methods but the consuming code was not fully updated, breaking bearer-token auth (every e2e failed with "Failed to generate bearer token") and the skyvault data-plane: - common/service_account/_utils.py: call get_bearer_token() (renamed from authentication_service_get_auth_token). - common/utils/_skyflow_messages.py: resolve the host SDK version via an importlib.metadata fallback so messages/metrics report the real version instead of 0.0.0. - skyvault/skyflow/service_account/__init__.py: re-export from common.service_account (skyvault's generated authentication client was removed by the regen), matching flowvault. - skyvault/skyflow/vault/controller/_vault.py: rewire record/token/query calls to the new generated method names; move file upload to the files client (upload_file_v2). - skyvault/skyflow/vault/client/client.py: add get_files_api(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…entifySourceType split The detect DataSourceType enum was split into per-SDK DeidentifySourceType and ReidentifySourceType, but the committed griffe baselines were not regenerated, so Contract Tests failed on public API drift. Regenerate both skyvault and flowvault baselines to record the intended enum change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…d-detect-v2-skyvault-flowvault
…d common auth Mirror the controller/auth migration in the unit tests so they exercise the methods the code now calls: - test__vault.py: rename the mocked generated methods to the new names (batch_operation/insert_records/update_record/delete_records/get_records/ execute_query/detokenize/tokenize), point file-upload at the files client (get_files_api + upload_file_v2), and drop byot from the batch_operation expectation (batch_operation no longer takes it; BYOT rides on per-record tokens). - service_account/test__utils.py and utils/test__utils.py: the public skyflow.service_account now routes through common, so patch common.service_account._utils.* and assert get_bearer_token on the mocked auth client instead of the removed authentication_service_get_auth_token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror skyflow-java's SK-3175 change. RedactionLevel was one common class carrying all four wire fields for every vault type. Replace it with an abstract RedactionRule base in common (exposing the four wire accessors) and a per-SDK RedactionLevel: - skyvault RedactionLevel carries entity_name + redaction_type - flowvault RedactionLevel carries token_group_name + redaction_pattern The reidentify mapper reads the wire fields through the rule's accessors, so each vault type only sends the fields its API supports. Refreshed the skyvault API contract baseline for the new class. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
common.vault.detect no longer exports a concrete RedactionLevel (it exports the RedactionRule base), so the reidentify mapping test uses a small concrete RedactionRule double instead of importing RedactionLevel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Detect V2 (
deidentify_string,deidentify_file,get_run,reidentify_string,reidentify_file,check_guardrails) to the Python SDKs. The implementation lives in the sharedcommonpackage and is re-exported from both flowvault and skyvault, with behaviour ported from the Java flowvault SDK (flowvault-release/26.10.1).Jira: SK-3075
What's included
common (
common/vault/detect,common/utils/enums/detect.py)SkyflowMessages.status=None, minimal client-side validation (server enforces the rest), auto-write of processed files for local-file inputs.flowvault
make_skyflow_class(detect_cls=DetectController)activatesclient.detect().VaultClientbuilds the common detect REST client and exposesget_strings_v2_api/get_files_v2_api/get_guardrails_v2_api.skyflow.vault.detectand the V2 enums; addssamples/detect_api/, a README section, and unit tests.skyvault
V1prefix,Skyflow→SkyflowAuth) and updates tests.Testing
common102 ✓,flowvault344 ✓,skyvault520 ✓;ruffclean.🤖 Generated with Claude Code